{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Scaling [WIP]\n",
    "\n",
    "Deals with normalizing range of features or independent variables of data. In ML, feature scaling is used to \n",
    "- ensure features contribute approximately proportionately to the final distance as opposed to one feature dominating; particularly important when euclidean distance is used\n",
    "- help gradient descent converge faster, thus helping with the optimization problem and speeding up the learning process\n",
    "- penalize coefficients appropriately when regularization is used as part of the loss function\n",
    "\n",
    "Source: [Wikipedia](https://en.wikipedia.org/wiki/Feature_scaling)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mean normalization\n",
    "\n",
    "Mean normalization is a method used in normalization. The formula is as follows:\n",
    "\n",
    "$$\n",
    "x^{\\prime}=\\frac{x-\\operatorname{average}(x)}{\\max (x)-\\min (x)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-0.5352, -0.0845,  0.0423,  0.1127,  0.4648])\n",
      "tensor([-1.4786, -0.2335,  0.1167,  0.3113,  1.2840])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "x = torch.Tensor([4, 36, 45, 50, 75])\n",
    "x_prime = (x - x.mean()) / (x.max() - x.min()) \n",
    "print(x_prime)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Standardization (Z-score Normalization)\n",
    "\n",
    "This can be used to ensure that the values of each feature in the data have zero-mean and unit-variance. The equation is as follows:\n",
    "$$\n",
    "x^{\\prime}=\\frac{x-\\operatorname{average}(x)}{\\operatorname{standard deviation}(x)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-1.4786, -0.2335,  0.1167,  0.3113,  1.2840])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "x = torch.Tensor([4, 36, 45, 50, 75])\n",
    "x_prime = (x - x.mean()) / x.std()\n",
    "print(x_prime)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Detailed Examples\n",
    "Here is an illustrated example with two features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f8a905653d0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAT50lEQVR4nO3dfYxcV33G8efBmHYJtFs1Vkk2XmypkasUtxiNEtBWiCK1TtKIWC6VEqlQKioLCiJQ6spBiFJUKW4j0bcgkEsiSkGBiqQrl6R1I20kCCKB9UswtkllIWi8RI1Ja4cII8Xh1z92nH2b3XnZO3PuOff7kVbZmbneORmtH5/7u79zriNCAID8vST1AAAA1SDQAaAQBDoAFIJAB4BCEOgAUIiXpnrjyy+/PLZs2ZLq7QEgS4cPH/5hRGzq9FqyQN+yZYtmZ2dTvT0AZMn291d7jZILABSCQAeAQhDoAFAIAh0ACkGgA0AhknW5jMr00TndeegJ/eDcBV05Pqa9O7dp146J1MMCgMoVHejTR+d0+/3HdeH5FyRJc+cu6Pb7j0sSoQ6gOEWXXO489MSLYX7Jhedf0J2Hnkg0IgAYnqID/QfnLvT1PADkrOiSy5XjY5rrEN5Xjo9lUVvPYYwA6qPoGfrends0tnHDkufGNm7Qb/7KJt1+/3HNnbug0EJtffroXJqBdnCp/l/nMQKol66Bbnuz7Ydtn7R9wvZtHY55k+3zto+1vz4ynOH2Z9eOCd2xe7smxsdkSRPjY7pj93Y9/J2zta+tU/8H0K9eSi4XJX0wIo7YfqWkw7YfioiTy477akTcVP0Q12fXjokVZYoPfPFYx2PrVFun/g+gX11n6BHxVEQcaX//I0mnJGVdyL1yfKyv51PIYYwA6qWvGrrtLZJ2SHqsw8tvsP247X+3/aur/Pk9tmdtz549e7b/0VZktdr63p3bEo1opRzGCKBeeu5ysf0KSfdJen9EPLvs5SOSXh0Rz9m+UdK0pKuX/4yIOCDpgCS1Wq0YdNDrdakEU+cOkhzGiLXRpYRRc0T3XLW9UdKXJR2KiI/3cPz3JLUi4oerHdNqtYIbXKBUy1cpS/NnWHfs3k6oY11sH46IVqfXeulysaS7JZ1aLcxtv6p9nGxf2/65zww+ZCBvdCkhhV5KLlOS3ibpuO1j7ec+JGlSkiLiU5LeKundti9KuiDpluhl6g8Uii4lpNA10CPiEUnucsxdku6qalDIX9Prx2utUgaGJfuVotNH5zS1f0Zb9z2gqf0zrKSsAVa50qWENLIOdIKjnqgfr75KuUlnKRi9rDfnWis4+IszXGuVVKgfz+u0ShkYpqxn6ARHGt3OjFjlCqSRdaATHGl0K6lQPwbSyDrQCY40up0ZUT8G0si6hs7y+DR6acmjfgyMXtaBLhEcKezdua3jsnbOjIC0sg90jB5nRkA9EegYCGdGQP0Q6KiFpm8VAFSBQEdyy7eavdTXLolQB/pAoCO5Xlb8MoMHuiPQkVy3vnZm8EBvsl5YhDJ0W/HLZl9Abwh0JNdtxS979gC9IdCRXLetAtizB+gNNXTUwlp97axMBXpDoKP2WJmKfjS5I4pARxZYmYpeNL0jiho6gGI0vSOKQAdQjKZ3RFFyQVaaXB9Fd73s1V8yZujIRrd7mQJNv4sZgY5sNL0+iu6afvtDSi7IRkn1UUpHw9Pkjihm6MhGKStGKR1hWAh0ZKOU+iilo+aZPjqnqf0z2rrvAU3tnxnaP96UXJCNUlaMllQ6QnejXOxEoCMrJdRHm95a1zS93MClKl1LLrY3237Y9knbJ2zf1uEY2/5726dtf8v26yodJVCQ3EtHoyoflGKUZ2S9zNAvSvpgRByx/UpJh20/FBEnFx1zg6Sr21/XSfpk+78Alsm5dNT0vVIGMcozsq6BHhFPSXqq/f2PbJ+SNCFpcaDfLOmzERGSHrU9bvuK9p8FsEyupaNRlg9KMcrtn/uqodveImmHpMeWvTQh6clFj8+0n1sS6Lb3SNojSZOTk30OFUBqXNDt3yjPyHoOdNuvkHSfpPdHxLODvFlEHJB0QJJarVYM8jMApMMF3cGM6oyspz502xs1H+afj4j7OxwyJ2nzosdXtZ8DUJDcL+iWrpcuF0u6W9KpiPj4KocdlPT2drfL6yWdp34OlKfpe6XUXS8llylJb5N03Pax9nMfkjQpSRHxKUkPSrpR0mlJP5b0h5WPFEAt5HpBtwl66XJ5RJK7HBOS3lPVoAAA/WMvFwAoBIEOAIUg0AGgEAQ6ABSC3RYzwN1tAPSCQK85NkMC0CtKLjXH3W0A9IoZuupd0mAzJAC9avwMve437C3lxsgAhq/xgV73kgabIQFp5XSHpsaXXOpe0qjr3W3qXKYCqpJbU0LjAz2H/Z3rthlSbr/kwKByu0NT40sulDT6V/cyFVCVup/BL9f4QGd/5/7l9ksODCq3poTGl1yk+pU06i6HMhVQhVHe4LkKjZ+ho3+UqdAUuZ3BM0NH3+raeQMMQ05n8AQ6BpLTLznQFJRcAKAQBDoAFIJAB4BCEOgAUAgCHQAKQaADQCEIdAAoBH3oAFChlFtLE+gAUJHUW0tTcgGAiqTeWppAB4CKpN5ampILssFt71B3qbeW7jpDt32P7adtf3uV199k+7ztY+2vj1Q/TDTdpdrk3LkLCi3UJut8w140T+qtpXspuXxG0vVdjvlqRLy2/fWx9Q8LWCp1bRLoRer907uWXCLiK7a3jGAswKpS1yaBXqXcWrqqGvobbD8u6QeS/jQiTnQ6yPYeSXskaXJysqK3RhOkrk0Ogpo/Rq2KLpcjkl4dEb8u6R8kTa92YEQciIhWRLQ2bdpUwVujKVLXJvtFzR8prDvQI+LZiHiu/f2DkjbavnzdIwMWSV2b7Bc1f6Sw7pKL7VdJ+p+ICNvXav4fiWfWPTJka1ilhpxue0fNHyl0DXTb90p6k6TLbZ+R9OeSNkpSRHxK0lslvdv2RUkXJN0SETG0EaPWUi99rosca/7IXy9dLrd2ef0uSXdVNiJkba1SQ5MCfe/ObUv+YZPqXfNHGVgpikpRaph36R8vulwwSgQ6KkWpYUFONX+Ugc25UKnc2gtLM310TlP7Z7R13wOa2j9Dm2TDMENHpSg1pMMFaRDoqBylhjRKuyDNStv+EehAIUq6IM3ZxmCooQOFWO3Cc44XpFlpOxgCHShESRekSzrbGCVKLsgS9dWVSrogTfvrYAh0ZIf66upKuSDNStvBUHJBdqivli+33TXrghk6skN9tRlKOdsYJWboyE5J3RxAlQh0ZKekbg6gSpRckJ2SujmAKhHoyBL1VWAlSi4AUAgCHQAKQaADQCEIdAAoBIEOAIUg0AGgEAQ6ABQi2z50tk8FgKWyDHS2TwWAlbIsubB9KgCslGWgs30qAKyUZaCzfSoArJRloLN9KgCslOVFUbZPBYCVuga67Xsk3STp6Yh4TYfXLenvJN0o6ceS3hERR6oe6HJsnwoAS/VScvmMpOvXeP0GSVe3v/ZI+uT6hwUA6FfXGXpEfMX2ljUOuVnSZyMiJD1qe9z2FRHxVFWDRP5SLgRjERqaoooa+oSkJxc9PtN+bkWg296j+Vm8JicnK3hr5CDlQjAWoaFJRtrlEhEHIqIVEa1NmzaN8q2RUMqFYCxCQ5NUEehzkjYvenxV+zlAUtqFYCxCQ5NUEegHJb3d814v6Tz1cyyWciEYi9DQJF0D3fa9kr4uaZvtM7bfaftdtt/VPuRBSd+VdFrSP0r646GNFllKuRCMRWhokl66XG7t8npIek9lI0JxUi4EYxEamsTzeTx6rVYrZmdnk7x3arTRoan43V8/24cjotXptSyX/ueMNjoMQw5Bye/+8GW5OVfOaKND1S4F5dy5CwotBOX00Xo1m/G7P3wE+oit1i43d+6CpvbP1O4vIeovl6CkhXT4CPQRW6tdrq4zK9RbLkFJC+nwEegj1qmNbrE6zqxyNn10TlP7Z7R13wPFngHlEpS0kA4fF0VHbHEb3VwmM6tcdboIt/dLj+ujB0/o/IXna3vxsF97d25b8v8p1TMo69xCmsNF5V7QtpjQ1P6ZjqE+MT6mr+17c4IRlWW1z3exsY0bdMfu7Vn+5V2slEBKYfk//FK9fy9oW6ypXGZWuerlTOdSiauOf3H7wQ1fBrfWReXcPlNq6Ant2jGhO3Zv18T4mKz5mXldZwU56rWGTImr2XK5qNwLZuiJMbMank5nQJ3U7eIhRuvK8bGOpbkcfy+YoaNYy8+AfuHlG7XxJV5yDCUulNR9wwwdRVt+BsTFQyxX5+6bftHlAgAZWavLhZILABSCQAeAQhDoAFAIAh0ACkGgA0AhaFsEkDVaURcQ6ACyxW3tlqLkAiBbudytaVQIdADZKmljrSoQ6ACylcvdmkaFQAeQrZI21qoCF0UBZKukjbWqkFWg054EYDnuKbAgm0CnPQkA1pZNoJd03z+gqTjLHq5sAp32JCBvnGUPX09dLravt/2E7dO293V4/R22z9o+1v76o6oHSnsSkDcWAQ1f10C3vUHSJyTdIOkaSbfavqbDoV+MiNe2vz5d8ThpTwIyx1n28PVScrlW0umI+K4k2f6CpJslnRzmwJajPamePjx9XPc+9qReiNAGW7det1l/uWt76mGhhq4cH9Nch/DmLLs6vQT6hKQnFz0+I+m6Dsf9ru03SvovSR+IiCeXH2B7j6Q9kjQ5Odn3YGlPqpcPTx/X5x797xcfvxDx4mNCHcvt3bltSQ1d4iy7alWtFP03SVsi4tckPSTpnzodFBEHIqIVEa1NmzZV9NZI5d7HVvybvebzaLZdOyZ0x+7tmhgfkyVNjI/pjt3bmaRVqJcZ+pykzYseX9V+7kUR8cyih5+W9NfrHxrq7oWIvp4HOMserl5m6N+UdLXtrbZfJukWSQcXH2D7ikUP3yLpVHVDRF1tsPt6HsBwdQ30iLgo6b2SDmk+qP8lIk7Y/pjtt7QPe5/tE7Yfl/Q+Se8Y1oBRH7det7mv5wEMlyPR6XGr1YrZ2dkk743q0OUCjJbtwxHR6vgagQ4A+Vgr0NkPHQAKkc1eLrlKsRkRGyABzUSgD1GKzYjYAAloLkouQ5RiMyI2QAKai0AfohSbEbEBEtBcBPoQpdjyl22GgeYi0IcoxZa/bDMM1Nf00TlN7Z/R1n0PaGr/jKaPznX/Q33gougQpdjyl22GsRgdT/UxioYFFhYBhVoeINL82Ro7HKYxtX+m437wE+Nj+tq+N/f8c1hYBDQQHU/1MoqGBQIdKBQdT/UyioYFAh0oFB1P9TKKhgUCHSgUHU/1Moo7NtHlAhRqvR1PdMgsqOqzGPYdmwh0oGCDBgh7Ai3I6bOg5AJgBTpkFgz6WQx7EVEnzNABrECHzIJBPotUs3pm6ABWGH/5xo7PN7FDZpBuoVRnOAQ6gCWmj87puZ9cXPH8xg1uZIfMIN1Cqc5wCHQAS9x56Ak9/9OVW4Jc9rKX1u4i4CgM0m6Yag0ANXQAS6w2izx/4fkRj6Q++u0W2rtzW8d9dIZ9hsMMHcASrDBdv1EsIuqEGTqAJVLNLksz7EVEnRDoAJZgT/18EehAw/SyjD3F7BLrR6ADDZLTMnb0r5GBzqZDaKq1FrzwdyB/jQt0ZihoMpb0l61xbYtsOoQmoyWxbD0Fuu3rbT9h+7TtfR1e/xnbX2y//pjtLZWPtCLMUNBk3PSibF0D3fYGSZ+QdIOkayTdavuaZYe9U9L/RcQvS/obSX9V9UCrwgwFTZZqwQtGo5ca+rWSTkfEdyXJ9hck3Szp5KJjbpb00fb3X5J0l21HxMoNIRJj0QSajpbEcvVScpmQ9OSix2faz3U8JiIuSjov6ReX/yDbe2zP2p49e/bsYCNeJ2YoAEo10i6XiDgg6YAktVqtZLN3ZigAStTLDH1O0uZFj69qP9fxGNsvlfTzkp6pYoAAgN70EujflHS17a22XybpFkkHlx1zUNIftL9/q6SZOtbPAaBkXUsuEXHR9nslHZK0QdI9EXHC9sckzUbEQUl3S/pn26cl/a/mQx8AMEI91dAj4kFJDy577iOLvv+JpN+rdmgAgH40bqUoAJTKqUrdts9K+n6SNx+NyyX9MPUgaoLPYgGfxQI+iwX9fBavjohNnV5IFuilsz0bEa3U46gDPosFfBYL+CwWVPVZUHIBgEIQ6ABQCAJ9eA6kHkCN8Fks4LNYwGexoJLPgho6ABSCGToAFIJAB4BCEOgVs32P7adtfzv1WFKzvdn2w7ZP2j5h+7bUY0rF9s/a/obtx9ufxV+kHlNKtjfYPmr7y6nHkprt79k+bvuY7dl1/Sxq6NWy/UZJz0n6bES8JvV4UrJ9haQrIuKI7VdKOixpV0Sc7PJHi2Pbki6LiOdsb5T0iKTbIuLRxENLwvafSGpJ+rmIuCn1eFKy/T1JrYhY9yIrZugVi4ivaH6DssaLiKci4kj7+x9JOqWVN0dphJj3XPvhxvZXI2dTtq+S9DuSPp16LKUh0DES7RuH75D0WOKhJNMuMxyT9LSkhyKiqZ/F30r6M0k/TTyOughJ/2n7sO096/lBBDqGzvYrJN0n6f0R8Wzq8aQSES9ExGs1f5OYa203riRn+yZJT0fE4dRjqZHfiIjXSbpB0nvaZduBEOgYqna9+D5Jn4+I+1OPpw4i4pykhyVdn3goKUxJeku7bvwFSW+2/bm0Q0orIuba/31a0r9KunbQn0WgY2jaFwLvlnQqIj6eejwp2d5ke7z9/Zik35L0naSDSiAibo+IqyJii+ZvhDMTEb+feFjJ2L6s3TAg25dJ+m1JA3fIEegVs32vpK9L2mb7jO13ph5TQlOS3qb5Wdix9teNqQeVyBWSHrb9Lc3f1vGhiGh8yx70S5Iesf24pG9IeiAi/mPQH0bbIgAUghk6ABSCQAeAQhDoAFAIAh0ACkGgA0AhCHQAKASBDgCF+H/SAjUyLUEVoAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# x_1 feature with 50 random values in range (0, 5)\n",
    "x_1 = torch.rand(50, 1) * 5\n",
    "x_2 = torch.rand(50, 1) * 3\n",
    "\n",
    "# scatter plot\n",
    "plt.scatter(x_1.numpy(), x_2.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate and subtract mean to get zero-mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f8a907042d0>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAATWElEQVR4nO3db4xc11nH8d8PN6kWilggFo3Xbm0gGCKMMB2lrfymtKnsVihJDYUECVpoZV40Aipk5CgVSBXIRpF4gYiorDZq+aMkVUm3Rgkybd0qEtAoYxzq/KnBRKTxNKLbpi5UdZs4fXixs9n17OzO7M6de+495/uRouzce7X36GbnyTnPec65jggBAPL3fakbAACoBwEfAApBwAeAQhDwAaAQBHwAKMQrUjdgLddcc03s3LkzdTMAoFVOnz79tYjYOuxcYwP+zp071e12UzcDAFrF9jNrnSOlAwCFIOADQCEI+ABQCAI+ABSCgA8AhWhslU5d5s/0dNfJc/rKxUvaNjujw/t365a9c6mbBQCVKzrgz5/p6Y4HzurSiy9JknoXL+mOB85KEkEfQHaKTuncdfLcy8F+yaUXX9JdJ88lahEATE/RAf8rFy9t6DgAtFnRKZ1tszPqDQnu22ZnWpHbb0MbATRH0T38w/t3a+aqLVccm7lqi37xp7fqjgfOqnfxkkLLuf35M700DR1iaf6hyW0E0CxFB/xb9s7p6ME9mpudkSXNzc7o6ME9+tyXFhqf22f+AcBGFZ3SkRaD/mAa5P33Pzb02ibl9pl/ALBRRffw17JtdmZDx1NoQxsBNAsBf4i1cvuH9+9O1KLV2tBGAM1SfEpnmKUUT5MrYNrQRoxGpRXq5IhI3YahOp1O8AIU5Gxwpbe0OEo7enAPQR+bZvt0RHSGnSOlAyRCpRXqRsAHEqHSCnUjh48kyF2vv9IbmIbse/jzZ3rad+yUdh15UPuOnWIlagOwSngRlVaoW9YBn8DSTOSuF6210ru0kQ7qk3VKZ73AwpdqutZL2ZC7XjZspTcwLVn38AksaYwaWbFKGEgj64BPYEljVMqG3DWQRtYBn8CSxqiRFblrII2sc/hsP5DGOOWG5K6B+mUd8CUCSwqH9+8eumUAIysgrewDPurHyApoJgI+poKRFdA8lUza2r7H9ldtP77Gedv+C9vnbX/R9i9UcV+UgxXTwOSqqtL5qKQD65x/m6Tr+v8ckvRXFd0XBWDFNFCNSgJ+RDws6fl1LrlZ0l/Hoi9ImrV9bRX3Rv7G2YqBEQAwWl11+HOSnl3x+UL/2BVsH7Ldtd1dWFioqWloulF1/YwAgPE0auFVRByPiE5EdLZu3Zq6OWiIUSum2YwNGE9dAb8naceKz9v7x4CRRq2YZs8kYDx1BfwTkn6zX63zBknfjIjnaro3Wm7UVgzsmQSMp5I6fNv3SnqTpGtsX5D0x5KukqSI+JCkhyS9XdJ5Sd+W9FtV3BflWK+un5W9wHgqCfgRcduI8yHpfVXcCxjEyl5sVKmv2GSlLbLAyl6Ma6mqa2lEuFTVJSn7v6FGVekAwLSVXNVFwAdQlJKrukjpICul5mYxvnHe15ArevjIBituMY6S34RHwEc2Ss7NYnwlv2KTlA6ykVNultTUdJVa1UUPH9nIZcUtqSlMCwEf2cglN0tqqkx1bPFNSgfZyGXFbU6pKYynrsVgBHxkJYfcbMllg6Vab1RX5d8zKR2gYdqemuLtYxtX16iOHj7QMG1OTZW8T80k6hrVEfCBBmpraqqu1ERu6trim4APoDJMOG9OXaM6Aj6AyjDhvHl1jOqYtAVQmbZPOOeOHj6AyrR5wrkEBHwAlWrrhHMJSOkAQCEI+ABQCAI+ABSCgA8AhSDgA0AhqNLJAG9HAjAOAn7LsVkVgHGR0mk53o4EYFz08MfQ5JQJm1UBGBc9/BGa/kLpXF7cDWD6CPgjND1lwmZVQFptesNXJQHf9gHb52yft31kyPl3216w/Vj/n/dWcd86ND1lcsveOR09uEdzszOypLnZGR09uCd5yqlNXwJgs5qeARg0cQ7f9hZJd0t6q6QLkh61fSIinhy49P6IuH3S+9WtDft7N22zKiqHUIq2veGrih7+DZLOR8TTEfGCpPsk3VzB720EUiYb1/Q0GFCVpmcABlUR8OckPbvi84X+sUG/bPuLtj9he8ewX2T7kO2u7e7CwkIFTZtcU1MmTda2LwGwWW0rmqirLPMfJN0bEd+1/TuSPibpzYMXRcRxScclqdPpRE1tG6lpKZOma0MaDKhCXS8fr0oVPfyepJU99u39Yy+LiK9HxHf7Hz8s6XUV3BcNRRoMpWhbBqCKHv6jkq6zvUuLgf5WSb++8gLb10bEc/2PN0l6qoL7oqF4zR1K0qYMwMQBPyIu275d0klJWyTdExFP2P6gpG5EnJD0u7ZvknRZ0vOS3j3pfdFsbfoSAKVwRGNS5VfodDrR7XZTNwMAWsX26YjoDDvHSlsAKAQBHwAKQcAHgEIQ8AGgEAR8ACgEAR8ACsEbrwCgRinfoEfAB4CapN46nJQOANQk9dbhBHwAqEnqrcNJ6SArKfOjwCiptw6nh49stO39oihP6q3DCfjIRur8KDBK6v3zSekgG6nzo8A4Um4dTsBHNlLnRzeK+QbUjZQOspE6P7oRzDcgBQI+spE6P7oRzDcgBVI6qN00UxltebUi8w1IgR4+akUqY9Fa8wpNnW9AHgj4qBWpjEVtmm9APkjpoFakMhYtpZ2o0kGdCPioVdtKJ6epLfMNyAcpHdSKVEZa82d62nfslHYdeVD7jp0qbu6kdPTwUStSGemk3osd6RHwUTtSGWmsN2Hexv8erFTeOAI+UIicJswZrWwOOXygEDnV/lPeuzkEfKAQOU2Y5zRaqRMpHWSJ/O5qOU2YU967OQR8ZIf87tpymTA/vH/3Ff+NpfaOVupUSUrH9gHb52yft31kyPlX2r6/f/4R2zuruC8wDPnd/LVpZ9QmmbiHb3uLpLslvVXSBUmP2j4REU+uuOw9kr4RET9p+1ZJfybp1ya9NzAM+d0y5DJaqVMVPfwbJJ2PiKcj4gVJ90m6eeCamyV9rP/zJyS9xbYruDewSk7VKECVqgj4c5KeXfH5Qv/Y0Gsi4rKkb0r60cFfZPuQ7a7t7sLCQgVNQ4lyqkYBqtSossyIOB4RnYjobN26NXVz0FLkd4HhqqjS6UnaseLz9v6xYddcsP0KST8k6esV3BsYivwusFoVPfxHJV1ne5ftqyXdKunEwDUnJL2r//OvSDoVEVHBvQEAY5q4hx8Rl23fLumkpC2S7omIJ2x/UFI3Ik5I+oikv7F9XtLzWvyfAgCgRpUsvIqIhyQ9NHDsj1b8/B1J76ziXgCAzWnUpC0AYHoI+ABQCAI+ABSCgA8Ahch2t0y2xwWAK2UZ8NkeFwBWyzKlw/a4ALBalgGf7XEBYLUsAz7b4wLAalkGfLbHBYDVspy0zellzQBQlSwDvsT2uAAwKMuUDgBgtWx7+GifVIvlWKSHUhDw0QipFsuxSA8lIaWDRki1WI5FeigJAR+NkGqxHIv0UBICPhoh1WI5FumhJAR8NEKqxXIs0kNJmLRFI6RaLMciPZTEEZG6DUN1Op3odrupm5EEZYIoFX/7k7N9OiI6w87Rw28YygQxDW0IpPztTx85/IahTBBVWwqkvYuXFFoOpPNneqmbdgX+9qePgN8wa5UD9i5e0r5jpxr3JUXztSWQUiI7fQT8hlmvHLCpPTM0W1sCKSWy00fAb5hhZYIrNbFn1nbzZ3rad+yUdh15MMtRVFsCKSWy00fAb5hb9s7p6ME9mlvny9i0nlmbDctvv//+x7Qzo+DflkC68m/fkuZmZ3T04J5GTNjm0imgSqeBlvby33fslHpDgnvTemZtNiy/vVSonEuVSJvWGjTxPRY5VQ8R8Bvs8P7dV/yhSc3smbXZqNHSUgqtbV/sQU0MpG2x3qR3254pKZ0Ga/IQNxfjjJZIoZWtLZPe45ioh2/7RyTdL2mnpP+W9KsR8Y0h170k6Wz/45cj4qZJ7lsSembTNWwUNYgUWtm2zc5kk1qdtId/RNJnI+I6SZ/tfx7mUkT8fP8fgj0aY3CS3APnSaGhLZPe45hoLx3b5yS9KSKes32tpM9HxKqnYPtbEfGqjfzukvfSQTpt2IIA9WvT38V6e+lMGvAvRsRs/2dL+sbS54HrLkt6TNJlScciYn6N33dI0iFJes1rXvO6Z555ZtNtA4ASTbR5mu3PSHr1kFN3rvwQEWF7rf97vDYierZ/XNIp22cj4r8GL4qI45KOS4s9/FFtAwCMb2TAj4gb1zpn+39sX7sipfPVNX5Hr//vp21/XtJeSasCPgBgeiadtD0h6V39n98l6VODF9j+Yduv7P98jaR9kp6c8L4AgA2aNOAfk/RW2/8p6cb+Z9nu2P5w/5qfkdS1/e+SPqfFHD4BHwBqNlEdfkR8XdJbhhzvSnpv/+d/kbRnkvsAACbH1goAstamksppI+ADyFZOG59Vgb10AGSrLW/7qgsBH0C2ctr4rAoEfADZasvbvupCwAeQrZw2PqsCk7YAstWmt33VIbuATwkWgJV4p8SyrAI+JVgAsLasAn5O754ESsQIfbqyCviUYAHtxQh9+rKq0qEEC2gvFklNX1YBnxIsoL0YoU9fVikdSrCa5wPzZ3XvI8/qpQhtsXXb63foT25h81Sstm12Rr0hwZ0RenWyCvgSJVhN8oH5s/rbL3z55c8vRbz8maCPQYf3774ihy8xQq9aVikdNMu9jzy7oeMo2y1753T04B7Nzc7IkuZmZ3T04B46cBXKroeP5ngphr+Hfq3jACP06aKHj6nZYm/oOIDpIuBjam57/Y4NHQcwXaR0MDVLE7NU6QDN4GhoPrXT6US3203dDABoFdunI6Iz7BwpHQAoBCmdxFJsFsUGVUCZCPgJpdgsig2qgHKR0kkoxWZRbFAFlIuAn1CKzaLYoAooFwE/oRTbObOFNFAuAn5CKbZzZgtpoLnmz/S079gp7TryoPYdO6X5M71Kfz+Ttgml2M6ZLaQxiKqtZqijoIKFV0DBBoOMtDjiY5fK+u07dmro+wDmZmf0z0fePPbvmdrCK9vvtP2E7e/ZHnqD/nUHbJ+zfd72kUnuCaA6VG01Rx0FFZPm8B+XdFDSw2tdYHuLpLslvU3S9ZJus339hPcFUAGqtpqjjoKKiQJ+RDwVEaO6AjdIOh8RT0fEC5Luk3TzJPcFUA2qtpqjjoKKOqp05iStfMXRhf6xVWwfst213V1YWKihaUDZqNpqjjre+DWySsf2ZyS9esipOyPiU5W1RFJEHJd0XFqctK3ydwNYbdKqLSp8FlX1HKb9xq+RAT8ibpzwHj1JK994sb1/DEADbDbIsC/TojY9hzpSOo9Kus72LttXS7pV0oka7gtgiqjwWTTJc5j2QqtBk5ZlvsP2BUlvlPSg7ZP949tsPyRJEXFZ0u2STkp6StLHI+KJyZoNIDUqfBZt9jksjQx6Fy8ptDwymGbQn7RK55MRsT0iXhkRPxYR+/vHvxIRb19x3UMR8VMR8RMR8aeTNhpAerPff9XQ46VV+Gy20inFCIm9dABs2PyZnr71ncurjl+1xcVV+Gy20inFCImAD2DD7jp5Ti9+b3Uh3Q9c/YrGTVRO22bLKVOsgWDzNAAbtlYv9JuXXqy5Jc2wmUqnw/t3D93HaJojJHr4ADaMFbqTq2Oh1SB6+AA2LEXvNEfTXmg1iIAPYMN4r0I7EfABXGHcbQLq7p1icgR8AC9r0zYB2DgC/hBsCIVSrbcYiO9A+xHwB9DDQcnYLiFvlGUOYEMolIxyy7wR8AfQw0HJeCFK3gj4A+jhoGQpFgOhPuTwB7CgBKWj3DJfBPwBLCgBkCsC/hD0cADkiBw+ABSCgA8AhSDgA0AhCPgAUAgCPgAUwhGr30vZBLYXJD2TsAnXSPpawvs3Cc9iGc9iGc9iWZOexWsjYuuwE40N+KnZ7kZEJ3U7moBnsYxnsYxnsawtz4KUDgAUgoAPAIUg4K/teOoGNAjPYhnPYhnPYlkrngU5fAAoBD18ACgEAR8ACkHAX4ftu2x/yfYXbX/S9mzqNqVi+522n7D9PduNLz+rmu0Dts/ZPm/7SOr2pGT7Httftf146rakZHuH7c/ZfrL/3fi91G0ahYC/vk9L+tmI+DlJ/yHpjsTtSelxSQclPZy6IXWzvUXS3ZLeJul6SbfZvj5tq5L6qKQDqRvRAJcl/UFEXC/pDZLe1/S/CwL+OiLinyLicv/jFyRtT9melCLiqYgo9U3uN0g6HxFPR8QLku6TdHPiNiUTEQ9Lej51O1KLiOci4t/6P/+fpKckNfpFGgT88f22pH9M3QgkMSfp2RWfL6jhX2zUy/ZOSXslPZK4Kesq/o1Xtj8j6dVDTt0ZEZ/qX3OnFodvf1dn2+o2zrMAcCXbr5L095J+PyL+N3V71lN8wI+IG9c7b/vdkn5J0lsi80ULo55FwXqSdqz4vL1/DIWzfZUWg/3fRcQDqdszCimdddg+IOkPJd0UEd9O3R4k86ik62zvsn21pFslnUjcJiRm25I+IumpiPjz1O0ZBwF/fX8p6Qclfdr2Y7Y/lLpBqdh+h+0Lkt4o6UHbJ1O3qS79ifvbJZ3U4sTcxyPiibStSsf2vZL+VdJu2xdsvyd1mxLZJ+k3JL25Hx8es/321I1aD1srAEAh6OEDQCEI+ABQCAI+ABSCgA8AhSDgA0AhCPgAUAgCPgAU4v8BO8//7NkcEssAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# subtract mean\n",
    "x = torch.concat([x_1, x_2], dim=1)\n",
    "x_new = x - x.mean(dim=0)\n",
    "\n",
    "# scatter plot x_new\n",
    "plt.scatter(x_new[:, 0].numpy(), x_new[:, 1].numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now normalize the variances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f8a906d1b90>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAV2UlEQVR4nO3dfYxc1XnH8d+v5iXbNOqS2OVlMbFpqZNGbgtdQRJHUV6gpqjCjpOoJH8E2kRu1KK2+cOVEVEqRa3sFKl/RKVNLYJK2ghoU3Cc4siBLFGkVlCWGDBvDgYltTdOcKDQRnECJk//mLswzM7szuzc9/P9SCvP3Hs99+z17uNzn/Occx0RAgC0389V3QAAQDkI+ACQCAI+ACSCgA8AiSDgA0AiTqq6AYOsXLky1qxZU3UzAKBR7r///h9GxKp++2ob8NesWaPZ2dmqmwEAjWL7u4P2kdIBgEQQ8AEgEQR8AEgEAR8AEkHAB4BE1LZKpyy798/pun0H9b3njuusyQlt27hOm8+fqrpZAJC7pAP+7v1zuua2Azr+4kuSpLnnjuua2w5IEkEfQOskndK5bt/Bl4P9vOMvvqTr9h2sqEUAUJykA/73njs+0nYAaLKkUzpnTU5ork9wP2tyohG5/Sa0EUB9JN3D37ZxnSZOXvGqbRMnr9C737RK19x2QHPPHVfoldz+7v1z1TS0j/nxhzq3EUC9JB3wN58/pR1b1mtqckKWNDU5oR1b1uvux4/VPrfP+AOAUSWd0pE6Qb83DfKJWx/oe2ydcvuMPwAYVdI9/EHOmpwYaXsVmtBGAPVCwO9jUG5/28Z1FbVooSa0EUC9JJ/S6Wc+xVPnCpgmtBFLo9IKZXJEVN2Gvqanp4MHoKDNemd6S527tB1b1hP0sWy274+I6X77SOkAFaHSCmUj4AMVodIKZcsl4Nu+0fbTth8esN+2P2v7kO2HbF+Qx3nRXLv3z2nDzhmt3X6HNuycSXLCGJVWKFtePfx/lHTpIvt/R9J52ddWSX+f03mXRGCpH2YJd1BphbLlEvAj4puSnl3kkE2SvhAd90iatH1mHudeDIGlnshddwya6c2ALYpSVlnmlKTDXe+PZNuOdh9ke6s6dwA655xzxj7pYoGFX6piLVZuSO76Ff1megNFqdWgbUTsiojpiJhetWrV2J9HYKnGUndW5K6BapQV8Ockre56f3a2rVAElmoslbIhdw1Uo6yAv0fSR7JqnbdKej4iji71l8ZFYKnGUndW5K6BauSSw7d9s6R3SVpp+4ikv5B0siRFxOck7ZV0maRDkn4s6ffzOO9SWH6gGos9WGYeuWugfLkE/Ij40BL7Q9If53GuURFYyrdt47q+SwZwZwVUi8XTkDvurIB6IuCjENxZAfVDwEcjsIwwMD4CPmqvdxnh+bp+SQR9YAQEfNTeMDOmuQMAlkbAR+0tVdfPHQAwnFotrQD0s9SMaRZjA4ZDwEftLTVjmjWTgOEQ8FF7Sy3FwJpJwHDI4aMRFqvrZ2YvMBwCPhqPmb0YVapVXQR8tAIzezGslKu6yOEDSErKVV0EfABJSbmqi5QOWiXV3CyGN8zzGtqKHj5aY6ln6QJS2k/CI+CjNVLOzWJ4KT9ik5QOWqNNuVlSU8VKtaqLHj5aoy0zbklNoSgEfLRGW3KzpKbStHv/nDbsnNHa7Xdow86ZQv6DJ6WD1mjLjNs2paYwnLImgxHw0SptyM2mXDaYqmEe8pMHUjpAzTQ9NVVGaqJtyrqro4cP1EyTU1Mpr1MzjrLu6gj4QA01NTVVVmqibcpa4puADyA3DDgvT1l3dQR8ALlhwHn5yrirY9AWQG6aPuDcdvTwAeSmyQPOKSDgA8hVUwecU0BKBwASQcAHgEQQ8AEgEQR8AEgEAR8AEpFLwLd9qe2Dtg/Z3t5n/1W2j9l+IPv6WB7nRQeLVQEYxthlmbZXSLpe0iWSjki6z/aeiHi059BbI+Lqcc+HV2OxKgDDyqOHf6GkQxHxVES8IOkWSZty+FwMgacjARhWHhOvpiQd7np/RNJFfY57v+13Svq2pE9ExOE+x9RSnR8ozWJVAIZV1qDtVyStiYhfl3SnpJv6HWR7q+1Z27PHjh0rqWmLq/sDpdvy4G4Axcsj4M9JWt31/uxs28si4pmI+Gn29gZJv9XvgyJiV0RMR8T0qlWrcmja+OqeMmGxKqBaTSqayCOlc5+k82yvVSfQXyHpw90H2D4zIo5mby+X9FgO5y1F3VMmdV2sqs5pMCAvTSuaGDvgR8QJ21dL2idphaQbI+IR25+WNBsReyT9ie3LJZ2Q9Kykq8Y9b1masL533RaratovAbBcTXvCVy45/IjYGxG/GhG/HBF/lW37VBbsFRHXRMRbIuI3IuLdEfF4HuctAymT0dU9DQbkpe4ZgF7MtF3C5vOntGPLek1NTsiSpiYntGPL+lr+710XTfslAJaraUUTrIc/hLqlTOquCWkwIA9lPXw8L/TwkTvSYEhF0zIA9PCRu7pWDgFFaFIGgICPQjTplwBIBSkdAEgEAR8AEkHAB4BEEPABIBEEfABIBAEfABJBwAeARFCHDwAlqnLpcAI+AJSk6qXDSekAQEmqXjqcgA8AJal66XBSOmgVHq2IOqt66XB6+GiN+fzo3HPHFXolP1rnh0ojLVUvHU7AR2tUnR8FllL1+vmkdNAaVedHgWFUuXQ4AR+tUXV+dFSMN6BspHTQGlXnR0fBeAOqQMBHa1SdHx0F4w2oAikdlK7IVEZTHq3IeAOqQA8fpSKV0TFoXKGu4w1oBwI+SkUqo6NJ4w1oD1I6KBWpjI75tBNVOigTAR+lalrpZJGaMt6A9iClg1KRyqjW7v1z2rBzRmu336ENO2eSGztJHT18lIpURnWqXosd1SPgo3SkMqqx2IB5E/89mKk8OgI+kIg2DZhzt7I85PCBRLSp9p/y3uUh4AOJaNOAeZvuVspESgetRH53oTYNmFPeuzwEfLQO+d3B2jJgvm3julf9G0vNvVspUy4pHduX2j5o+5Dt7X32n2r71mz/vbbX5HFeoB/yu+3XpJVR62TsHr7tFZKul3SJpCOS7rO9JyIe7Trso5L+JyJ+xfYVkj4j6ffGPTfQD/ndNLTlbqVMefTwL5R0KCKeiogXJN0iaVPPMZsk3ZS9/pKk99p2DucGFmhTNQqQpzwC/pSkw13vj2Tb+h4TESckPS/pDb0fZHur7Vnbs8eOHcuhaUhRm6pRgDzVqiwzInZFxHRETK9atarq5qChyO8C/eVRpTMnaXXX+7Ozbf2OOWL7JEm/KOmZHM4N9EV+F1gojx7+fZLOs73W9imSrpC0p+eYPZKuzF5/QNJMREQO5wYADGnsHn5EnLB9taR9klZIujEiHrH9aUmzEbFH0ucl/ZPtQ5KeVec/BQBAiXKZeBUReyXt7dn2qa7XP5H0wTzOBQBYnloN2gIAikPAB4BEEPABIBEEfABIRGtXy2R5XAB4tVYGfJbHBYCFWpnSYXlcAFiolQGf5XEBYKFWBnyWxwWAhVoZ8FkeFwAWauWgbZse1gwAeWllwJdYHhcAerUypQMAWKi1PXw0T1WT5Zikh1QQ8FELVU2WY5IeUkJKB7VQ1WQ5JukhJQR81EJVk+WYpIeUEPBRC1VNlmOSHlJCwEctVDVZjkl6SAmDtqiFqibLMUkPKXFEVN2Gvqanp2N2drbqZlSCMkGkip/98dm+PyKm++2jh18zlAmiCE0IpPzsF48cfs1QJoi8zQfSueeOK/RKIN29f67qpr0KP/vFI+DXzKBywLnnjmvDzpna/ZKi/poSSCmRLR4Bv2YWKwesa88M9daUQEqJbPEI+DXTr0ywWx17Zk23e/+cNuyc0drtd7TyLqopgZQS2eIR8Gtm8/lT2rFlvaYW+WWsW8+syfrltz9x6wNa06Lg35RA2v2zb0lTkxPasWV9LQZs29IpoEqnhubX8t+wc0ZzfYJ73XpmTdYvvz1fqNyWKpEmzTWo43Ms2lQ9RMCvsW0b173qB02qZ8+syZa6W5pPoTXtF7tXHQNpUyw26N20a0pKp8bqfIvbFsPcLZFCS1tTBr2HQQ+/5uiZFavfXVQvUmhpO2tyojWpVXr4SFrvILl79pNCQ1MGvYdBDx/J676LasISBChXkwa9l8LiaQDQIostnkZKBwASMVbAt/1623fafiL787QBx71k+4Hsa8845wQALM+4Pfztkr4eEedJ+nr2vp/jEfGb2dflY54TALAM4wb8TZJuyl7fJGnzmJ8HACjIuAH/9Ig4mr3+vqTTBxz3Gtuztu+xvXnQh9nemh03e+zYsTGbBgDotmRZpu27JJ3RZ9e13W8iImwPKvl5Y0TM2T5X0oztAxHxZO9BEbFL0i6pU6WzZOsBYAmU2r5iyYAfERcP2mf7B7bPjIijts+U9PSAz5jL/nzK9jcknS9pQcAHgDy1aeGzPIyb0tkj6crs9ZWSvtx7gO3TbJ+avV4paYOkR8c8LwAsqSlP+yrLuAF/p6RLbD8h6eLsvWxP274hO+bNkmZtPyjpbkk7I4KAD6BwbVr4LA9jLa0QEc9Iem+f7bOSPpa9/k9J68c5DwAsR5sWPssDM20BtFabFj7LA4unAWitNi18lofWBXxKsAB045kSr2hVwKcECwAGa1XAb9OzJ4EUcYderFYFfEqwgObiDr14rarSGVRqlWoJFtAkTJIqXqsCPiVYQHNxh168VqV0KMGqn0/uPqCb7z2slyK0wtaHLlqtv9zMPDwsxCSp4rUq4EuUYNXJJ3cf0D/f898vv38p4uX3BH302rZx3aty+BJ36HlrVUoH9XLzvYdH2o60bT5/Sju2rNfU5IQsaWpyQju2rKcDl6PW9fBRHy9F/0caDNoOcIdeLHr4KMwKe6TtAIpFwEdhPnTR6pG2AygWKR0UZn5gliodoB4cNc2nTk9Px+zsbNXNAIBGsX1/REz320dKBwASQUqnYlUsFsUCVUCaCPgVqmKxKBaoAtJFSqdCVSwWxQJVQLoI+BWqYrEoFqgC0kXAr1AVyzmzhDSQLgJ+hapYzpklpIH62r1/Tht2zmjt9ju0YeeMdu+fy/XzGbStUBXLObOENHpRtVUPZRRUMPEKSFhvkJE6d3ysUlm+DTtn+j4PYGpyQv+x/T1Dfw4TrwD0RdVWfZRRUEHABxJG1VZ9lFFQQcAHEkbVVn2UUVBBwAcSRtVWfZTxxC+qdICEjVu1RYVPR17XoegnfhHwgcQtN8iwLlNHk64DKR0Ay0KFT8c416HoiVa96OEDWBYqfDqWex2quDOghw9gWSZ//uS+21Or8FlupVMVd0gEfAAj271/Tj/6yYkF209e4eQqfJZb6VTFHdJYAd/2B20/YvtntvtO5c2Ou9T2QduHbG8f55wAqnfdvoN68WcLl2V57Skn1W6gsmjLLaesYg7EuDn8hyVtkfQPgw6wvULS9ZIukXRE0n2290TEo2OeG0BFBvVCnz/+YsktqYflVDpt27iu7zpGRd4hjdXDj4jHImKphNOFkg5FxFMR8YKkWyRtGue8AKrFDN3xlTHRqlcZVTpTkg53vT8i6aJ+B9reKmmrJJ1zzjnFtwzAslTRO22joida9Voy4Nu+S9IZfXZdGxFfzrMxEbFL0i6pszxynp8NID88V6GZlgz4EXHxmOeYk7S66/3Z2TYANTTsMgFl904xvjJSOvdJOs/2WnUC/RWSPlzCeQGMqEnLBGB045Zlvs/2EUlvk3SH7X3Z9rNs75WkiDgh6WpJ+yQ9JulfIuKR8ZpdrLKnOwN1wXIJ7TZWDz8ibpd0e5/t35N0Wdf7vZL2jnOustDDQcpYLqHdmGnbgx4OUka5ZbsR8HvQw0HKeCBKuxHwe9DDQcqqmAyE8rA8cg8mlCB1lFu2FwG/BxNKALQVAb8PejgA2ogcPgAkgoAPAIkg4ANAIgj4AJAIAj4AJMIR9Vx23vYxSd9dxl9dKemHOTcnD7RrNLRrNLRrNG1u1xsjYlW/HbUN+MtlezYiBj5QvSq0azS0azS0azSptouUDgAkgoAPAIloY8DfVXUDBqBdo6Fdo6Fdo0myXa3L4QMA+mtjDx8A0AcBHwAS0fiAb/s624/bfsj27bYnBxx3qe2Dtg/Z3l5Cuz5o+xHbP7M9sMzK9ndsH7D9gO3ZGrWr7Ov1ett32n4i+/O0Ace9lF2rB2zvKbA9i37/tk+1fWu2/17ba4pqy4jtusr2sa5r9LES2nSj7adtPzxgv21/NmvzQ7YvKLpNQ7brXbaf77pWnyqpXatt32370ex38U/7HFPMNYuIRn9J+m1JJ2WvPyPpM32OWSHpSUnnSjpF0oOSfq3gdr1Z0jpJ35A0vchx35G0ssTrtWS7Krpefy1pe/Z6e79/x2zfj0q4Rkt+/5L+SNLnstdXSLq1Ju26StLflvXzlJ3znZIukPTwgP2XSfqqJEt6q6R7a9Kud0n69zKvVXbeMyVdkL1+naRv9/l3LOSaNb6HHxFfi4gT2dt7JJ3d57ALJR2KiKci4gVJt0jaVHC7HouI2j35fMh2lX69ss+/KXt9k6TNBZ9vMcN8/93t/ZKk99p2DdpVuoj4pqRnFzlkk6QvRMc9kiZtn1mDdlUiIo5GxLey1/8n6TFJvQ/gKOSaNT7g9/gDdf5X7DUl6XDX+yNaeIGrEpK+Zvt+21urbkymiut1ekQczV5/X9LpA457je1Z2/fY3lxQW4b5/l8+JutwPC/pDQW1Z5R2SdL7szTAl2yvLrhNw6jz79/bbD9o+6u231L2ybNU4PmS7u3ZVcg1a8QTr2zfJemMPruujYgvZ8dcK+mEpC/WqV1DeEdEzNn+JUl32n4865lU3a7cLdau7jcREbYH1Qu/Mbte50qasX0gIp7Mu60N9hVJN0fET23/oTp3Ie+puE119S11fp5+ZPsySbslnVfWyW3/gqR/k/RnEfG/ZZyzEQE/Ii5ebL/tqyT9rqT3RpYA6zEnqbunc3a2rdB2DfkZc9mfT9u+XZ3b9rECfg7tKv162f6B7TMj4mh26/r0gM+Yv15P2f6GOr2jvAP+MN///DFHbJ8k6RclPZNzO0ZuV0R0t+EGdcZGqlbIz9O4uoNsROy1/Xe2V0ZE4Yuq2T5ZnWD/xYi4rc8hhVyzxqd0bF8q6c8lXR4RPx5w2H2SzrO91vYp6gyyFVbhMSzbr7X9uvnX6gxA960oKFkV12uPpCuz11dKWnAnYvs026dmr1dK2iDp0QLaMsz3393eD0iaGdDZKLVdPXney9XJD1dtj6SPZJUnb5X0fFf6rjK2z5gfd7F9oTrxsOj/tJWd8/OSHouIvxlwWDHXrOwR6ry/JB1SJ9f1QPY1XzlxlqS9Xcddps5o+JPqpDaKbtf71Mm7/VTSDyTt622XOtUWD2Zfj9SlXRVdrzdI+rqkJyTdJen12fZpSTdkr98u6UB2vQ5I+miB7Vnw/Uv6tDodC0l6jaR/zX7+/kvSuUVfoyHbtSP7WXpQ0t2S3lRCm26WdFTSi9nP1kclfVzSx7P9lnR91uYDWqRqreR2Xd11re6R9PaS2vUOdcbuHuqKW5eVcc1YWgEAEtH4lA4AYDgEfABIBAEfABJBwAeARBDwASARBHwASAQBHwAS8f+qaAZlDF+ZfQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# normalize variance\n",
    "x_new_normalized = x_new / torch.std(x_new, unbiased=False)\n",
    "\n",
    "# scatter plot x_new_normalized\n",
    "plt.scatter(x_new_normalized[:, 0].numpy(), x_new_normalized[:, 1].numpy())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here is one more basic example where we reproduce the same results from this [scikit-learn tutorial](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.5000, 0.5000])\n",
      "tensor([[-1., -1.],\n",
      "        [-1., -1.],\n",
      "        [ 1.,  1.],\n",
      "        [ 1.,  1.]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([[0.0, 0.0], [0.0, 0.0], [1.0, 1.0], [1.0, 1.0]])\n",
    "x_new = x - x.mean(dim=0)\n",
    "print(x.mean(dim=0))\n",
    "x_new_normalized = x_new / torch.std(x_new, unbiased=False)\n",
    "print(x_new_normalized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "- [Wikipedia - Feature Scaling](https://en.wikipedia.org/wiki/Feature_scaling)\n",
    "- [Wikipedia - Standard Deviation](https://en.wikipedia.org/wiki/Standard_deviation)\n",
    "- [Normalizing Inputs by DeepLearning.AI](https://www.youtube.com/watch?v=FDCfw-YqWTE&ab_channel=DeepLearningAI)\n",
    "- [How To Calculate the Mean and Standard Deviation — Normalizing Datasets in Pytorch](https://towardsdatascience.com/how-to-calculate-the-mean-and-standard-deviation-normalizing-datasets-in-pytorch-704bd7d05f4c)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.12 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "d4d1e4263499bec80672ea0156c357c1ee493ec2b1c70f0acce89fc37c4a6abe"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
